Spiral Recurrent Neural Network for Online Learning
نویسندگان
چکیده
Autonomous, self* sensor networks require sensor nodes with a certain degree of “intelligence”. An elementary component of such an “intelligence” is the ability to learn online predicting sensor values. We consider recurrent neural network (RNN) models trained with an extended Kalman filter algorithm based on real time recurrent learning (RTRL) with teacher forcing. We compared the performance of conventional neural network architectures with that of spiral recurrent neural networks (Spiral RNN) a novel RNN architecture combining a trainable hidden recurrent layer with the “echo state” property of echo state neural networks (ESN). We found that this novel RNN architecture shows more stable performance and faster convergence.
منابع مشابه
Conditional prediction of time series using spiral recurrent neural network
Frequently, sequences of state transitions are triggered by specific signals. Learning these triggered sequences with recurrent neural networks implies storing them as different attractors of the recurrent hidden layer dynamics. A challenging test and also useful for application is conditional prediction of sequences giving just the trigger signal as an input and letting the recurrent neural ne...
متن کاملA Solution to the Problem of Extrapolation in Car Following Modeling Using an online fuzzy Neural Network
Car following process is time-varying in essence, due to the involvement of human actions. This paper develops an adaptive technique for car following modeling in a traffic flow. The proposed technique includes an online fuzzy neural network (OFNN) which is able to adapt its rule-consequent parameters to the time-varying processes. The proposed OFNN is first trained by an growing binary tree le...
متن کاملTowards Real-World Applications of Online Learning Spiral Recurrent Neural Networks
Distributed intelligent systems like self-organizing wireless sensor and actuator networks are supposed to work mostly autonomous even under changing environmental conditions. This requires robust and efficient self-learning capabilities implementable on embedded systems with limited memory and computational power. We present a new solution called Spiral Recurrent Neural Networks (SpiralRNN) wi...
متن کاملA Recurrent Neural Network Model for solving CCR Model in Data Envelopment Analysis
In this paper, we present a recurrent neural network model for solving CCR Model in Data Envelopment Analysis (DEA). The proposed neural network model is derived from an unconstrained minimization problem. In the theoretical aspect, it is shown that the proposed neural network is stable in the sense of Lyapunov and globally convergent to the optimal solution of CCR model. The proposed model has...
متن کاملDesigning stable neural identifier based on Lyapunov method
The stability of learning rate in neural network identifiers and controllers is one of the challenging issues which attracts great interest from researchers of neural networks. This paper suggests adaptive gradient descent algorithm with stable learning laws for modified dynamic neural network (MDNN) and studies the stability of this algorithm. Also, stable learning algorithm for parameters of ...
متن کامل